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Abstract —We study signal recovery on graphs based on two 
sampling strategies: random sampling and experimentally de¬ 
signed sampling. We propose a new class of smooth graph 
signals, called approximately bandlimited. We then propose two 
recovery strategies based on random sampling and experimentally 
designed sampling. The proposed recovery strategy based on 
experimentally designed sampling uses sampling scores, which is 
similar to the leverage scores used in the matrix approximation. 
We show that while both strategies are unbiased estimators for the 
low-frequency components, the convergence rate of experimentally 
designed sampling is much faster than that of random sampling 
when a graph is irregulaiQ We validate the proposed recovery 
strategies on three specific graphs: a ring graph, an Erdos-Renyl 
graph, and a star graph. The simulation results support the 
theoretical analysis. 

L Introduction 

We consider sampling and recovery within the framework 
of signal processing on graphs, which studies signals with an 
underlying complex structure ID, El- The framework models 
that underlying structure by a graph and signals by graph 
signals, generalizing concepts and tools from classical discrete 
signal processing. The task of sampling and recovery is one 
of the most critical topics in the signal processing community. 
As the bridge connecting sequences and functions, classical 
sampling theory shows that a bandlimited function can be 
perfectly recovered from its sampled sequence if the sampling 
rate is high enough. The interest in sampling and recovery of 
graph signals has increased in the last few years El, El, El- 
In El, authors proposed an algorithm to recover graph signals 
that have small variation based on random sampling. In El, 
m, authors proposed a sampling theory for graph signals and 
show perfect recovery for bandlimited graph signals based on 
experimentally designed sampling. 

In this paper, we propose a new class of graph signals, 
called approximately bandlimited. We then propose two recov¬ 
ery strategies based on random sampling and experimentally 
designed sampling, and bound the recovery error for the class 
of approximately bandlimited graph signals. We show that 
the proposed recovery strategies are unbiased estimators for 
low-frequency components and that experimentally designed 
sampling outperforms random sampling in terms of the conver¬ 
gence rate when a graph is irregular. We validate both recovery 
strategies on three specific graphs: a ring graph, an Erdos-Renyi 
graph, and a star graph. The simulation results support the 
theoretical analysis. 

'Due to lack of space, the proofs of lemmas and theorems are omitted and 
will be included in an expanded version of the paper. 


II. Discrete Signal Processing on Graphs 

We now briefly review discrete signal processing on 
graphs 0, which lays a foundation for the proposed work. 
We consider a graph G = (V, A), where V = {uq, ..., pat-i} 
is the set of nodes and A G is the graph shift, or a 

weighted adjacency matrix. As the most basic filter defined 
on this graph, the graph shift represents the connections of 
the graph G, which can be either directed or undirected. The 
edge weight An,m between nodes and Vm is a quanti¬ 
tative expression of the underlying relation between the nth 
and the mth node, such as a similarity, a dependency, or a 
communication pattern. To guarantee that the filtered signal 
is properly scaled for comparison in the original one 0, we 
normalize the graph shift, such that |Ainax(A)| = 1. Once the 
node order is fixed, the gr^h signal can be written as a vector, 
X = [ccq, tci,..., Xn-i] G K.^. The Jordan decomposition 
of A is 0 

A = VAU, (1) 

where the generalized eigenvectors of A form the columns of 
matrix V , U = (the norm of each row of U is normalized 
to one), and A G is the block diagonal matrix of 

corresponding eigenvalues Aq, ... A^v-i of A (1 = Aq > Ai > 
..., > Xn -1 > ~1)- The graph Fourier transform of x G 
is 

X = U X. (2) 

The inverse graph Fourier transform is x = V x = 
Xk'Vk, where is the fcth column of V and Xk is 
the fcth component in x. The vector x in 0 represents the 
signal’s expansion in the eigenvector basis and describes the 
frequency components of the graph signal x. The inverse graph 
Eourier transform reconstructs the graph signal by combining 
graph frequency components. When A represents an undirected 
graph, we have U = V^, and both U and V are orthonormal. 
In general, V may not be orthonormal; to restrict its behavior, 
we assume that 

||x ||2 < ||Vx||^ < a 2 ||x ||2 , for all x G (3) 

where ai,a 2 > 0, that is. Visa Riesz basis with stability 
constants ai, a 2 0. The eigenvalues Aq, ... Xn-i of A, 
represent frequencies on the graph 0. 

III. Problem Eormulation 

We now review two standard classes of graph signals, and 
propose a new one, which connects the first two. We next 


describe the sampling and recovery strategies. In this way, we 
show the connection between this work and the previous work: 
graph signal inpainting and sampling theory on graphs. 

A. Graph Signal Model 

We focus on smooth graph signals, that is, the signal 
coefficient at each node is close to the signal coefficients of 
its neighbors. In literature Q, Q, two classes of graph signals 
have been introduced to measure the smoothness on graphs. 

Definition 1. A graph signal x S is globally smooth on a 
graph A G with parameter rj >0, when 

||x-Ax ||2 < ??||x|| 2 . (4) 

Denote this class of graph signals by GSa(p)- 

Since we normalized the graph shift such that |Amax(A)| = 
1; when p > 4, all graph signals satisfy (IHi. While the 
recovery of globally smooth graph signals has been studied 
in @ (leading to graph signal inpainting), global smoothness 
is a general requirement, making it hard to provide further 
theoretical insight ||9l. 

Definition 2. A graph signal x G is bandlimited on a 
graph A with parameter K G {0, 1, • • • , — 1}, when the 

graph frequency components x satisfies 

Xk = 0 for all k> K. 

Denote this class of graph signals by BLA(Ar). 

Note tht the original definition requires x be AT-sparse, which 
unnecessarily promotes smoothness Q. While the recovery of 
bandlimited graph signals has been studied in I?) (leading to 
sampling theory on graphs), the bandlimited requirement is a 
restrictive requirement, making it hard to use in the real world 
applications. We thus propose a third class that relaxes it, but 
still promotes smoothness. 

Definition 3. A graph signal x G is approximately 
bandlimited on a graph A with parameters /3 > 1 and p, > 0, 
when there exists aArG{0,l,'-',iV—1} such that its graph 
Fourier transform x satisfies 

N-l 

+ < pMI. 

k=K 

Denote this class of graph signals by ASiLp^{K,fi,p). 

We see that BLA(Ar) is a subset of ABLA(Ar, /3,/i) with 
/i = 0,/3 = 0. The approximately bandlimited class allows for 
a tail after the first K frequency components. The parameter 
p controls the shape of the tail; the smaller the p, the smaller 
the energy contribution from the high-frequency components. 
The parameter controls the speed of energy decaying; the 
larger the /3, the larger the penalty on the high-frequency 
components. The class of ABLA(Ar) is similar to the ellipsoid 
constraints in cni, where all the graph frequency components 
are considered in the constraints; thus, ABLA(Ar) provides 
more flexibility for the low-frequency components. 

The following theorem shows the relationship between 
kELp,{K, (3, p) and GSa(p)- 


Theorem 1. ASLx{K, (3, p) is a subset of GSa(p), when 


T? > 1 - Aa- 



GSa(? 7) is a subset of A^Ij j^{K, (3, p), when 


M > 


1 + {N - 1)2/3 

(1 — \k)oii ^ 


From Theorem [T] we see that when choosing proper param¬ 
eters, GSa(? 7) is a subset of A^\jj^{K, [3, p). 


B. Sampling & Recovery 

We consider the procedure of sampling and recovery as 
follows: we sample M coefficients in a graph signal x G 
with noise to produce a noisy sampled signal y G that is. 


y = T-x -f e = X 7 W -f e, (5) 

where e ~ A/’(0, tr^ M = (A4o, • • • , Mm-i) denotes 

the sequence of sampled indices, called sampling set, with 
Mi G {0,1, • • • ,N — 1} and |A4| = M, -km is the noiseless 
sampled coefficients, and the sampling operator T* is a linear 
mapping from jq ]^m 

vk, ^ ~ (6) 

’■3 \ 0, otherwise. 

We then interpolate y to get x' G which recovers x either 
exactly or approximately. We consider two sampling strate¬ 
gies: random sampling means that sample indices are chosen 
from from - ,N — 1} independently and randomly; 

and experimentally design sampling means that sample indices 
can be chosen beforehand. It is clear that random sampling is 
a subset of experimentally design sampling. 

IV. Recovery Strategy 

We now propose two recovery strategies based on ran¬ 
dom sampling and experimentally designed sampling. In Sec¬ 
tion HlFAl we showed that a graph signal is smooth when 
its energy is mainly concentrated in the low-frequency com¬ 
ponents. For example, for the class BLA(Ar), all the energy 
is concentrated in the first K frequency components and the 
graph signal can be perfectly recovered by using those first 
K frequency components. The recovery strategies we propose 
here follow this intuition, by providing unbiased estimators for 
the low-frequency components. 

A. Recovery Strategy based on Random Sampling 
We consider the following recovery strategy. 

Algorithm 1. We sample a graph signal \M \ times. Each time, 
we choose a node i independently and randomly, and take a 
measurement We then recover the original graph signal by 
using the following two steps: 

' ' iGM 

* \ '' r 

Xi = 

k<.K, 







where x* is the ith component of the recovered graph signal x*. 

Algorithm [T] aims to estimate the first k frequency compo¬ 
nents, and reconstruct the original graph signal based on these 
graph frequency components. The only tuning parameter in 
Algorithm [T] is the bandwidth k. To show the performance of 
Algorithm [U for recovering the low-frequency components, we 
have the following results. 

Denote V(k) be the first k columns of the inverse graph 
Fourier transform matrix V, and U(„) be the first k rows of 
the graph Fourier transform matrix U. 

Lemma 1. Algorithm[T]with bandwidth k provides an unbiased 
estimator of the first k frequency components, that is. 

Ex* = V(^)U(^)X, for all X, 

where x* is the result of Algorithm [T] 

The advantage of Algorithm [T] is its efficiency, that is, we 
only need the first k eigenvectors involved in the computation, 
which is appealing for large-scale graphs. The disadvantage is 
that when the main energy of an original graph signal is not 
concentrated in the first k frequency components, the recovered 
graph signal has a large bias. 


Theorem 2. For x S ABL(iT,/3,/r), let x* be the result of 
Algorithm [T] with bandwidth k > AT, we have, 

a 2 (maxj x'^ -f 


Ellx* -xll 


< 


Q2F ||x||^ 
k2/3 


\M\ 


Af U 


(ft) 


|2 

If ' 


where a 2 is the stability constant of V in ([2i, is the noise 
level in (|5]), and H-Hp is the Frobenius norm. 


Due to the limited space, we do not show the proof here. 
The main idea follows from the bias-variance tradeoff. The first 
term is the bias term, and the second terms is the variance 
term. Since Algorithm [T] can recover the first k frequency 
components on expectation, the bias comes from the other 
{N—k) frequency components, which can be bounded from the 
definition of ABLfiT,/r,/3) when k > K. The variance term 
depends on ||U(ft)||p, which represents the graph structure. 

B. Recovery Strategy based on Experimentally Designed Sam¬ 
pling 

We consider the following recovery strategy. 


Algorithm 2. We sample a graph signal \ Jvi \ times. Each time, 
we choose a node with probability Wi = ||ui II 2 / IIW' II 2 ’ 

where is the ith column of U(k), and take a measurement 
Di. We then recover the original graph signal by using the 
following two steps; 

i&M 

X* = 

k<.K 

where x* is the ith component of the recovered graph signal x*. 

Similarly to Algorithm [T] Algorithm |2] aims to estimate the 
first K frequency components, and reconstructs the original 


graph signal based on these graph frequency components. The 
difference comes from the normalization factor. In Algorithm[T] 
the contribution from each measurement is normalized by 
a constant, the size of the graph; and in Algorithm |2l the 
contribution from each measurement is normalized based on 
the norm of the corresponding column in U(k), called sampling 
scores. Sampling scores are similar to leverage scores used in 
the matrix approximation El, where the goal is to evaluate the 
contribution from each column to approximating matrix. Note 
that leverage scores use the norm square, ||ui|| 2 , and we use 
the norm, ||ui|| 2 . When we use the norm square as sampling 
scores, the performance is the same with the random sampling. 

We can show that Algorithmic] is also an unbiased estimator 
for recovering the low-frequency components. 

Lemma 2. Algorithmic] with bandwidth k provides an unbiased 
estimator of the first k frequency components, that is. 


Ex* = V(«)U(^)X, for all X , 

where x* is the result of Algorithmic] 


Theorem 3. For x G ABLa(J^, /?, /r), let x* be the result of 
Algorithm 12 with bandwidth k > K, we have. 


Ellx* -xll 


< 


a2A I 


, 2/3 


a2(maxj x^ + 

\M\ 


U(«) 


2 

2,1 ■ 


The main idea also follows from the bias-variance trade¬ 
off. We see that Algorithms [T] and |2 have the same bias 
by recovering the first k frequency components on expecta¬ 
tion. When each column of U(„) has roughly similar energy, 
W||U(k)||^ and ||U(k )||2 2 similar. However, when the 
energy is concentrated on a few columns, W ||U(k) ||^ is much 
larger than ||U(k)|| 2 j^, in other words. Algorithm |2 has a 
significant advantage over Algorithm [T] when the associated 
graph structure is irregular. 


C. Convergence Rates 

To discriminate the proposed recovery strategies, we propose 
two types of graphs, and compare the convergence rates of 
Algorithms [T] and |2 for each type of these two. 

Definition 4. A graph A G is type-1, when 

I I = for alH, j = 0,1, • • • , — 1, 


where U are the graph Fourier transform matrix of A. 


For a type-1 graph, each element in U has roughly similar 
magnitudes, that is, the energy evenly spreads to each element 
in U. Some examples are discrete-time graphs, discrete-space 
graphs, and unweighted circulant graphs El- Based on The¬ 
orems 12 and 12 we conclude as follows. 

Corollary 1. Let A G be a type-1 graph, for the class 

KSLa{K,P,p). 

« Let X * be the results given by Algorithm [T] with the 
bandwidth k > K, we have 

e (|| x *- x || 2 ) < CN\M\~^, 






where C > 0, and the rate is achieved when n is in the 
order of and upper bounded by TV; 

■ Let X* be the results given by Algorithm |2] with the 
bandwidth k > K, we have 

]e(||x*-XII2) < CN\M\~^, 

where C > 0, and the rate is achieved when n is in the 
order of and upper bounded by N. 


When |Af| ^ we set K = and then the bias term is 
zero, and both upper bounds are actually CA^jAf We see 
that Algorithms [T] and |2] have the same convergence rate, that 
is, experimentally designed sampling does not perform better 
than random sampling for the type-1 graphs. 


Definition 5. A graph A G is type-2 with parameter 

Kq > 0, when 


h 


(K) 

'J'C 


< a 
1 



for all K > Kq, 


where = \JYl!k=Q i, T indexes the largest K elements 

in h, indexes the other {N — K) elements, and a > 0 is a 
constant. 


A type-2 graph requires the sampling scores to be approxi¬ 
mately sparse. When we take the first K > Kq rows to form 
a submatrix, the energy in the submatrix concentrates in a few 
columns. 

Based on Theorems |2] and [3 we conclude the following. 

Corollary 2. Let A G be a type-2 graph with parameter 

Kq, for the class ABLa{K, P, p,). 

• Let X* be the results given by Algorithm [1] with the 
bandwidth k > K, we have 

e (|| x *- x || 2 ) < CN\M\~^, 

where C > 0, and the rate is achieved when k is in the 
order of and upper bounded by N-, 

m Let X* be the results given by Algorithm |2] with the 
bandwidth k > max{Ar, we have 

e(||x*-x||^) < CN\M\- 2/3 + 2-7 < C'N\M\ 2/3 + 1, 


where C > 0, the rate is achieved when k, is in the order 
of |7Vd|i/(2/3-t2-7) and upper bounded by N, and 


7 e 


[max{l, 2/3 -I- 2 
max{l 


log|-M| T 
logmaxjAT, iVo} 
(2/3 + 2)logTV 
’ (logA^ + log|Af|)^^' 


Similarly to the type 1 graphs, when \Ai\ ^ N, we set 
K = N, and then the bias term is zero, and both upper 
bounds are CN\J^\~^. We see that Algorithm |2] has a larger 
convergence rate than Algorithm [T] that is, experimentally de¬ 
signed sampling exhibits much better performance than random 
sampling for the type-2 graph. The advantages follow from 
that, for type-2 graphs, ||U(k )||2 is in the order of k , and 

11 11 2 ’ 

A^ ||U(k)||p is in the order of Nk. 


We propose Definition |5] to obtain the asymptotic results, 
however, it is too strict to model real-world graphs. When 
considering a small sample size and a small bandwidth, we 
just need the requirement in Definition |5] holds for some 
K = Kq < N. We call those graphs as the general type-2 
graphs. Simulations shows that scale-free scales belong to the 
general type-2 graphs. 

D. Relation to Graph Signal Inpainting 

Graph signal inpainting aims at recovering globally smooth 
graph signals based on random sampling 0. It solves the 
following optimization problem, 

X* = argmin llx — Axil? , (7a) 

subject to ll'kx —y ||2 < cr^, (7b) 

where is noise level, y is a vector representation of the noisy 
measurements Q, and T* is the sampling operator (|6]l. Graph 
signal inpainting focuses on recovery in the vertex domain, 
and the proposed recovery strategies focus on recovery in 
graph spectral domain. The optimum of (|2ll guarantees that the 
recovered graph signal is close to the measurements at given 
nodes, but Algorithms [T] and |2] guarantee the recovery of the 
low-frequency components. 

E. Relation to Sampling Theory on Graphs 

Sampling theory on graphs aims at recovering bandlimited 
graph signals based on both the random sampling and the 
experimentally designed sampling Q. It solves the following 
optimization problem, 

X* = argmin^gBLA(K) 11^^ “ yWl = V(if)(«' V(if))+y, (8) 

where T* is the sampling operator y is a vector rep¬ 
resentation of the noisy measurements ©, and (•)+ is the 
pseudo-inverse. When the original graph signal is bandlimited, 
X G BLA(Ar), it is clear that the result of ® is an unbi¬ 
ased estimator of x. When the original graph signal is not 
bandlimited, the result of © is a biased estimator of the first 
K frequency components, because the signal belonging to the 
other frequency band also projects onto the first K components. 
In a sense of recovering the low-frequency components, 
needs fewer samples, but Algorithms [T] and |2] are more reliable. 

V. Experimental Results 

In this section, we compare the empirical performance of 
Algorithms [ 1 ] and 12] on three specific graphs: a ring graph, an 
Erdos-Renyi graph, and a star graph. 

Eor each graph A, we generate 50 graph signals by the 
following two steps. We first generate the graph frequency 
components as 

^ / - Ar(l,0.5) if k<K, 
if k>K. 

We then normalize x to have norm one, and obtain x = V x. 
It is clear that x G ABLa(A', /3, p), where K — IQ and /3 = 1. 
During the sampling, we simulate the noise e M{Q, 0.01). In 
the recovery, we set the bandwidth k to 10 for both algorithms. 
We consider the following three graphs. 
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(a) Ring graph with 4-nearest neighbor . 


(b) Erdos-Renyi graph. 


(c) Star graph. 


Fig. 1: Comparison of recovery error of Algorithm [T] (blue curve) and (red curve). MSE indicates the mean square etTor. 


Ring Graph with fc-nearest Neighbors. We consider a 
graph with each node connecting to its A:-nearest neighbors. 
The eigenvectors are similar to the discrete cosine transform 
and the energy evenly spreads to each element in U d, which 
follows Definition |4] Based on Corollary [T] we expect that 
Algorithm |2] has a similar performance with Algorithm [T] In 
the simulation, the ring graph has 10,000 nodes, and each node 
connects to its 4 nearest neighbors. 

Erdos-Renyi Graph. We consider a random graph where 
each pair of nodes is connected with some probability, also 
known as an Erdos-Renyi graph ini- Since the maximum 
value of eigenvectors of an Erdos-Renyi graph is bounded by 
lfT4l . the energy also spreads to each element in V, 
which follows Definition |4] Based on Corollary [T] we expect 
that Algorithm |2] has a similar performance with Algorithm [T| 
In the simulation, the Erdos-Renyi graph has 10,000 nodes, and 
each pair of nodes is connected with probability of 0.01, that 
is, each node has 100 neighbors on expectation. 

Star Graph. We consider a graph with a central node 
connecting to all other nodes, known as the star graph. The 
simulations show that star graphs approximately follows Def¬ 
inition |5] Based on Corollary |2l we expect that Algorithm 
outperforms Algorithm [T] In the simulation, the star graph has 
10,000 nodes. 

Results. Eigure[T]compares the performances between Algo¬ 
rithms [Hand |2] averaged over 50 tests. The blue curve represents 
Algorithm (Tj the red curve represents Algorithm [2] and the 
black dotted line represented the linear approximation by the 
true first K frequency components. We see that both algorithms 
converges to the linear approximation by the first K frequency 
components, which supports the results in Lemmas [H and [2] 
Eor two type-1 graphs, including the ring graph with 4-nearest 
neighbors and an Erdos-Renyi graph. Algorithms [T] and [2] 
provide similar results; however, for the star graph, Algorithm|2] 
performs much better than Algorithm [T] which supports the 
results in Corollaries [T]and|2l 

VI. Conclusions 

We proposed a new class of smooth graph signals, called 
approximately bandlimited, and we then proposed two recov¬ 


ery strategies based on random sampling and experimentally 
designed sampling. We showed that both strategies are unbiased 
estimators for the low-frequency components, and experimen¬ 
tally designed sampling outperforms random sampling when a 
graph is irregular. We validate the recovery strategies on three 
specific graphs: a ring graph, an Erdos-Renyi graph, and a star 
graph. The simulation results support the theoretical analysis. 

References 

[1] D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, 
“The emerging field of signal processing on graphs: Extending high¬ 
dimensional data analysis to networks and other irregular domains,” IEEE 
Signal Process. Mag., vol. 30, pp. 83-98, May 2013. 

[2] A. Sandryhaila and J. M. F. Moura, “Big data processing with signal 
processing on graphs,” IEEE Signal Process. Mag., vol. 31, no. 5, pp. 
80 - 90, 2014. 

[3] I. Z. Pesenson, “Sampling in Paley-Wiener spaces on combinatorial 
graphs,” Trans. Amer. Math. Soc., vol. 360, no. 10, pp. 5603-5627, May 
2008. 

[4] A. Anis, A. Gadde, and A. Ortega, “Towards a sampling theorem for 
signals on arbitrary graphs,” in Proc. IEEE Int. Conf. Acoust., Speech 
Signal Process., May 2014, pp. 3864 — 3868. 

[5] X. Wang, P. Liu, and Y. Gu, “Local-set-based graph signal reconstruc¬ 
tion,” IEEE Trans. Signal Process., 2015, To appear. 

[6] S. Chen, A. Sandryhaila, J. M. F. Moura, and J. Kovacevic, “Signal 
recovery on graphs,” IEEE Trans. Signal Process., 2014, To appear. 

[7] S. Chen, R. Varma, A. Sandryhaila, and J. Kovacevic, “Discrete signal 
processing on graphs: Sampling theory,” IEEE Trans. Signal Process., 
2015, Submitted. 

[8] M. Vetterli, J. Kovacevic, and V. K. Goyal, Foundations 
of Signal Processing, Cambridge University Press, 2014, 
http ://w w w.fourierandwavelets. org/. 

[9] J. Sharpnack and A. Singh, “Identifying graph-structured activation 
patterns in networks,” in Proc. Neural Information Process. Syst., 
Vancouver, Dec. 2010, pp. 2137-2145. 

[10] I. M. Johnstone, Minimax Bayes, Asymptotic Minimax and Sparse 
Wavelet Priors, Statistical Decision Theory and Related Topics V. 
Springer, 1994. 

[11] P. Drineas, M. Magdon-Ismail, M. W. Mahoney, and D. Woodruff, “Fast 
approximation of matrix coherence and statistical leverage,” Journal of 
Machine Learning Research, vol. 13, pp. 3475-3506, 2012. 

[12] A. Sandryhaila, J. Kovacevic, and M. Piischel, “Algebraic signal 
processing theory: 1-D nearest-neighbor models,” IEEE Trans. Signal 
Process., vol. 60, no. 5, pp. 2247-2259, May 2012. 

[13] M. Newman, Networks: An Introduction, Oxford University Press, 2010. 

[14] L. V. Tran, V. H. Vu, and K. Wang, “Sparse random graphs: Eigenvalues 
and eigenvectors,” Random Struct. Algorithms, vol. 42, no. 1, pp. 110- 
134, 2013. 















